This invention is related to the field of computer systems and, more particularly, to computer systems in which certain write transactions may be considered completed by a source upon transmission (i.e., posted write transactions).
Generally, personal computers (PCs) and other types of computer systems have been designed around a shared bus system for accessing memory. One or more processors and one or more input/output (I/O) devices are coupled to memory through the shared bus. The I/O devices may be coupled to the shared bus through an I/O bridge which manages the transfer of information between the shared bus and the I/O devices, while processors are typically coupled directly to the shared bus or are coupled through a cache hierarchy to the shared bus.
Unfortunately, shared bus systems suffer from several drawbacks. For example, the multiple devices attached to the shared bus present a relatively large electrical capacitance to devices driving signals on the bus. In addition, the multiple attach points on the shared bus produce signal reflections at high signal frequencies which reduce signal integrity. As a result, signal frequencies on the bus are generally kept relatively low in order to maintain signal integrity at an acceptable level. The relatively low signal frequencies reduce signal bandwidth, limiting the performance of devices attached to the bus.
Lack of scalability to larger numbers of devices is another disadvantage of shared bus systems. The available bandwidth of a shared bus is substantially fixed (and may decrease if adding additional devices causes a reduction in signal frequencies upon the bus). Once the bandwidth requirements of the devices attached to the bus (either directly or indirectly) exceeds the available bandwidth of the bus, devices will frequently be stalled when attempting access to the bus, and overall performance of the computer system including the shared bus will most likely be reduced.
On the other hand, distributed memory systems lack many of the above disadvantages. A computer system with a distributed memory system includes multiple nodes, two or more of which are coupled to different memories. The nodes are coupled to one another using any suitable interconnect. For example, each node may be coupled to each other node using dedicated lines. Alternatively, each node may connect to a fixed number of other nodes, and transactions may be routed from a first node to a second node to which the first node is not directly connected via one or more intermediate nodes. A memory address space of the computer system is assigned across the memories in each node.
In general, a xe2x80x9cnodexe2x80x9d is a device which is capable of participating in transactions upon the interconnect. For example, the interconnect may be packet based, and the node may be configured to receive and transmit packets as part of a transaction. Generally speaking, a transaction is a series of packets. A xe2x80x9crequesterxe2x80x9d or xe2x80x9csourcexe2x80x9d node initiates a transaction directed to a xe2x80x9ctargetxe2x80x9d node by issuing a request packet. Each packet which is part of the transaction is communicated between two nodes, with the receiving node being designated as the xe2x80x9cdestinationxe2x80x9d of the individual packet. When a packet ultimately reaches the target node, the target node accepts the information conveyed by the packet and processes the information internally. Alternatively, a node located on a communication path between the requester and target nodes may relay the packet from the requester node to the target node.
In addition to the original request packet, the transaction may result in the issuance of other types of packets, such as responses, probes, and broadcasts, each of which is directed to a particular destination. For example, upon receipt of the original request packet, the target node may issue broadcast or probe packets to other nodes in the processing system. These nodes, in turn, may generate responses, which may be directed to either the target node or the requester node. If directed to the target node, the target node may respond by issuing a response back to the requester node.
Distributed memory systems present design challenges which differ from the challenges in shared bus systems. For example, shared bus systems regulate the initiation of transactions through bus arbitration. Accordingly, a fair arbitration algorithm allows each bus participant the opportunity to initiate transactions. The order of transactions on the bus may represent the order that transactions are performed (e.g. for coherency purposes). On the other hand, in distributed systems, nodes may initiate transactions concurrently and use the interconnect to transmit the transactions to other nodes. These transactions may have logical conflicts between them (e.g. coherency conflicts for transactions involving the same address) and may experience resource conflicts (e.g. buffer space may not be available in various nodes) since no central mechanism for regulating the initiation of transactions is provided. Accordingly, it is more difficult to ensure that information continues to propagate among the nodes smoothly and that deadlock situations (in which no transactions are completed due to conflicts between the transactions) are avoided.
For example, certain deadlock conditions may occur in known I/O systems, such as the Peripheral Component Interconnect (PCI) I/O system, unless packets associated with a xe2x80x9cpostedxe2x80x9d write transaction are allowed to pass other traffic not associated with a posted write transaction. Generally speaking, a posted write transaction is considered complete by the requester when the write request and corresponding data are transmitted by the requester (e.g., by a source interface). A posted write transaction is thus effectively completed at the requester. However, because the requester is not directly aware of when the posted write transaction is actually completed by the target, more ordering support for posted transactions may be required in hardware. If sufficient hardware is available to support ordering, the requester may issue additional requests while the packet or packets of the posted write transaction travel to the target, with the assumption that such additional requests will complete after the initial posted transaction completes.
In contrast, a xe2x80x9cnon-postedxe2x80x9d write transaction is not considered complete by the requester until the target (e.g., a target interface) has completed the non-posted write transaction. The target generally transmits an acknowledgement to the requester when the non-posted write transaction is completed. Such acknowledgements, however, consume interconnect bandwidth and must be received and accounted for by the requester. Non-posted write transactions may be issued, for example, when the requester needs to know that previous transactions have been completed before issuing subsequent transactions.
In a computer system having a distributed memory system, memory requests (e.g., read and write transactions) originating from I/O nodes may need to be properly ordered with respect to other pending memory operations to preserve memory coherency within the computer system and to satisfy ordering requirements expected by the I/O nodes. For example, memory operations may need to be completed in the order in which they were generated to preserve memory coherency within the computer system and to satisfy I/O ordering requirements. It would thus be desirable to have a computer system implementing a system and method for tracking the progress of posted write transactions. Such a system and method would facilitate proper ordering of I/O memory operations.
A computer system is presented which implements a system and method for tracking the progress of posted write transactions. In one embodiment, the computer system includes a processing subsystem and an input/output (I/O) subsystem. The processing subsystem includes multiple processing nodes interconnected via coherent communication links. Each processing node may include a processor preferably executing software instructions (e.g., a processor core configured to execute instructions of a predefined instruction set). The I/O subsystem includes at least one I/O node, and may include multiple I/O nodes coupled via non-coherent communication links one after another in series or daisy chain fashion. Each I/O node may embody one or more I/O functions (e.g., modem, sound card, etc.).
The multiple processing nodes may include a first processing node and a second processing node, wherein the first processing node includes a host bridge, and wherein a memory is coupled to the second processing node. The host bridge translates packets moving between the processing subsystem and the I/O subsystem. One of the I/O nodes is coupled to the first processing node via a non-coherent communication link. The I/O node coupled to the first processing node may generate a non-coherent write transaction to store data within the memory, wherein the non-coherent write transaction is a posted write transaction. The I/O node may dispatch the non-coherent write transaction directed to the host bridge.
As a posted write transaction, the non-coherent write transaction may be completed by the I/O node when the non-coherent write transaction is dispatched by the I/O node. I/O transaction packets may travel in one of multiple separate virtual channels within the I/O subsystem. For example, the non-coherent write transaction may travel within a posted command virtual channel of the I/O subsystem, wherein the posted command virtual channel is separate from a non-posted command virtual channel of the I/O subsystem.
The host bridge may respond to the non-coherent write transaction by translating the non-coherent write transaction to a coherent write transaction, and dispatching the coherent write transaction within the processing subsystem. The coherent write transaction is directed to the second processing node. The second processing node may respond to the coherent write transaction by dispatching a target done response directed to the host bridge.
The second processing node may issue the target done response when the coherent write transaction has at least reached a xe2x80x9cpoint of coherencyxe2x80x9d within the processing subsystem. The coherent write transaction may reach the point of coherency when the coherent write transaction is: (i) ordered within the second processing node with respect to other requests pending within the second processing node, and (ii) a coherency state with respect to the coherent write transaction has been established in the other processing nodes.
The host bridge may include translation logic for translating the non-coherent write transaction to the coherent write transaction. The host bridge may also include a data buffer for storing transaction data associated with the non-coherent write transaction and the corresponding coherent write transaction. The transaction data may include data identifying the non-coherent write transaction as a posted write transaction, and data identifying the I/O node coupled to the first processing node as the source of the non-coherent write transaction. The data identifying the I/O node coupled to the first processing node as the source of the non-coherent write transaction may include a unit ID value extracted from the non-coherent write transaction by the host bridge. The unit ID value may be one of several unit IDs assigned to the I/O node coupled to the first processing node.
In one embodiment of a method for use in the computer system described above, the I/O subsystem dispatches the non-coherent write transaction to the host bridge. The non-coherent posted write is considered complete by the I/O subsystem as soon as the non-coherent posted write is dispatched to the host bridge. The host bridge translates the non-coherent write transaction to a coherent write transaction, and dispatches the coherent write transaction to a target within the processing subsystem. The target may be, for example, a memory controller within the second processing node and coupled to the memory. The target dispatches a target done response directed to the host bridge in response to the coherent write transaction. The target may dispatch the target done response in response to the coherent write transaction at least reaching a point of coherency within the computer system as described above. The target done response may thus signify that the coherent write transaction has at least reached the point of coherency within the processing subsystem, allowing subsequent order-dependent coherent transactions to be issued by the host bridge. The I/O node coupled to the second processing node may be a source of the non-coherent write transaction. The non-coherent and coherent write transactions and the target done response may be dispatched as one or more packets.
The above described system and method for tracking the progress of posted write transactions allows an I/O node within the I/O subsystem to garner the performance advantages offered by posted write transactions, while at the same time facilitating proper ordering of the posted write transactions.